Games Instructor : Alex Slivkins Homework 1 : bandits with IID rewards

Problem 1: rewards from a small interval. Consider a version of the problem in which all the realized rewards are in the interval [12 , 1 2 + ] for some ∈ (0, 1 2). Define versions of UCB1 and Successive Elimination attain improved regret bounds (both logarithmic and root-T) that depend on the . Hint: Use a more efficient version of Hoeffding Inequality in the slides from the first lecture. It is OK not to repeat all steps from the analysis in the class as long as you explain which steps in the analysis are changed. Comments after due date: The confidence radius and the √ T regret bound can be improved by the factor . The log(T )/∆ regret bound can be improved by the factor of 2. (The 2 improvement is a little surprising; but ∆ ≤ , so, in some sense, one factor of cancels out.) An alternative solution is to transform all rewards from r to (r − 12)/ . Then we obtain the problem with [0, 1] rewards, and regret in the original problem is times regret in the transformed problem. (And ∆ in the original problem is times the ∆ in the transformed problem, hence the 2 improvement in the log(T ) regret bound.) Problem 2: instantaneous regret. Recall: instantaneous regret at time t is defined as ∆(at).